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ABSTRACT 

Teachers, districts, and states working with the 
Northwest Regional Education Laboratory have been using a six-trait 
analytical scoring assessment model for student writing. The six 
traits are: (1) ideas; (2) organization; (3) voice; (4) word choice; 
(5) sentence fluency; and (6) conventions. For the last 4 years, the 
Northwest Regional Education Laboratory has been training teachers to 
teach students to be assessors of writing using the same six traits. 
This study was conducted to investigate the usefulness of this 
approach. Six classrooms of fifth graders participated, 67 in the 
treatment (integration) group and 65 in the control (observation) 
group. Teachers were taught scoring strategies and were visited by 
project staff to assist in implementing them. Pretest scores were 
very similar for both groups. The treatment group gained the most on 
those traits receiving the most emphasis in instruction, in 
proportion to the amount of time spent on them and in the order in 
which they were introduced; their performance on other traits 
improved slightly. Students in the control group improved slightly on 
two traits and remained at nearly the same level on the other four. 
One figure and two tables present study findings. Appendixes contain 
the student scoring guide and a scoring guide for adult scorers. 
(SLD) 
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The Impact of Training Students to Be 
Self-Assessors of Writing 



Introduction 

For the last ten years teachers, districts, and states have been using a six-trait analytical scoring 
assessment model for tracking and analyzing the quality of student writing. The six-trait model 
uses professional judgment to rate pieces of writing along six dimensions: 

• Ideas. Ideas are the heart of the message, the content of the piece, the ' -nn theme, 
together with all the details that enrich and develop that theme. 

• Organization, Organization is the internal structure of a piece of writing, the thread of 
central meaning, the pattern that holds everything together. 

• Voice. Voice is the writer coming through the words, the sense that a real person is 
speaking to us and cares about the message. 

• Word Choice. Word choice is the use of rich, colorful, precise language that 
communicates not just in a functional way, but in a way that moves and enlightens the 
reader. 

• Sentence Fluency. Sentence fluency is the rhythm and flow of the language, the sound of 
word patterns, the way in which the writing plays to the ear — not just to the eye. 

• Conventions. Conventions are the mechanical correctness of the piece — grammar, 
spelling, paragraphing (the use of indents), capitalization, and punctuation. 

(A student version of the scoring guide for these traits is provided in Appendix A. The scoring 
guide used by adult raters is provided in Appendix B.) 

During the course of using the six-trait model for large-scale assessment, we have noticed that 
those doing the scoring seem to learn a lot about writing. In addition to learning about how 
students write, they become better writers themselves. They also report that they are better able 
to teach writing, because now they understand what they need to teach. 

For the last four years we have been training teachers to teach students to be assessors of writing 
(both their own writing and that of others) using the six-traits. The rationale is that if raters of 
writing learn so much about writing from systematically analyzing and rating it, why not pass this 
power on to students? Anecdotal evidence from teachers strongly supports the power of this 
notion; teachers tell us how much more comfortable they feel teaching writing and how much 
better student writing has become since systematically gearing instruction to the six-trait model. 

The purpose of this study was to collect evidence to support or refute the power of this approach. 
The specific research question to be answered was: 
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Does the writing of students who have direct instruction on assessing writing using the six- 
trait analytical model improve more than the writing of students who do not have such 
instruction? 



Method 

Six classrooms of fifth graders were recruited for participation in the study to represent a range of 
learning contexts (rural/urban, size of district, size of school, student expenditures) and student 
types. Classrooms were located in different buildings and different school districts. Classrooms 
were randomly assigned to either a "treatment" (integration) or a "control" (observation) 
condition. There were 67 students in the integration group and 65 students in the observation 
group. 

Participating teachers in the integration sites (plus any other teachers in the school that wished to 
attend) received a one-day training session in October 1992, on writing assessment and 
integrating the six-trait model into instruction. This training included definition of each of the six 
traits, opportunity to practice assessing student writing for each of the six traits, instruction in 
how to teach the traits to students, and plans for "mini-lessons" for each trait. 

Teachers received an extensive set of training materials to serve as a resource throughout the 
year. Materials included scoring guides written for teachers and for students at the fifth grade 
level, plus sets of classroom activities designed to help students develop skills in each trait. In 
addition, teachers were provided with sample student essays to share and discuss with their 
students and strategies for successfully teaching students to become self-assessors using the six- 
trait model. 

Each treatment teacher was visited eight times between November 1992 and April 1993 to assist 
in implementing the strategies overviewed during the initial training session. The first visit was a 
general overview session for students and the start of training on the trait of Ideas. The next two 
visits continued training on the trait of Ideas. This was followed by two visits on each of the traits 
of Organization and Voice. The final visit was a wrap-up and debrief. The final three traits 
(Word Choice, Sentence Fluency and Conventions) were mentioned but not addressed in any 
depth. 

At treatment sites, teachers provided extensive instruction directly to students on the traits. This 
instruction included: (1) sharing and evaluating (as a group) samples of literature to illustrate 
strong examples of each trait, (2) evaluating the work of students from other classrooms and 
discussing how to improve the writing on the trait under consideration, (3) mini-lessons in which 
teachers covered skills with respect to each trait (e.g., how to generate ideas, different kinds of 
organization, how to write good leads, etc.), and (4) application of the traits to students' c wn 
writing (either self-assessment or peer review). Students were given their own scoring guides to 
work from. In one classroom, parenis also received instruction in the model so that they could 
provide reinforcement at home. 
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Each control teacher was visited three times between November 1992 and April 1993 to 
determine how writing instruction occurred. Staff simply noted the kinds and amount of writing 
instruction provided by teachers, and asked questions about their normal strategies for writing 
instruction. All control sites used a process approach to writing in which students had an 
opportunity to write more than one draft and sometimes to share their writing with the teacher or 
with peers to obtain feedback. The amount of writing students did in the classroom in a week 
varied from approximately one-half hour to more than three hours. Control teachers were not 
observed writing with students. However, they did provide instruction in prewriting 
(brainstorming, listing, etc.). In one classroom, students had extensive opportunities to use word 
processing for structured assignments, and to help each other with revision and editing. Students 
were sometimes in: tructed to do additional writing outside of class. No control teacher provided 
direct instruction on the qualities of good writing. 

Thus, the writing instruction provided in the control and treatment classrooms was substantially 
different. (Note: At the end of the study, teachers and other interested staff in the control 
schools also received a one-day training on writing assessment and the packet of instructional 
materials.) 

Data Source and Study Design 

Prior to the beginning of the study (November 1992), and at the end of the study (May 1993) 
students wrote essays under the following conditions: Students had three 45-minute periods to 
write an essay on an assigned topic. Students had time to prewrite, create a rough draft, read 
over the rough draft and revise, sdit, and produce a final copy. Students were allowed to use 
dictionaries or other written references during testing, but could not confer with the teacher or 
with each other. Two prompts were used-expository (to elicit informational writing) and 
narrative (to elicit stories). The topics were written to allow a variety of interpretations and 
approaches. (See Figure 1.) 
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Figure 1 
Writing Prompts 

Expository: Think of the most valuable thing you own which was not bought in a store. 
Explain why it is important to you. 

Narrative: You won't believe it, but here's what happened... 



At pretest time, the two prompts were randomly distributed in each classroom. At posttest time 
students responded to the prompt they did not get at pretest time. This procedure was (1) used to 
eliminate memory effects that come from repeating prompts, yet (2) control for differences in 
difficulty of prompts. The same prompts were used pre and post to eliminate differences in 
prompt difficulty, yet different students received the prompts to eliminate memory effects. 
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Random distribution served to reduce self-selection effects and ensure that, on average, student 
ability didn't interact with prompt. 

Pretest information was used to plan instruction in the three treatment classrooms. Since these 
pretests were also used at the end of the study to measure impact on students, they were rescored 
along with the posttests by other raters. For the impact study, we controlled for possible rater 
bias by removing paper identification, mixing pre and posttest essays, and randomly distributing 
them to experienced adult raters. Each paper received scores on each trait from each of two 
raters. 

Data were analyzed using six (one for each trait) repeated measures (matched pre and posttest 
scores) analyses of variance. 

Results 

Pre and posttest means are shown in Table 1 and statistical results are shown in Table 2. 



Table 1 

Means of Treatment Groups on Pre and Posttests 



Trait 


Group 


Pre 


Post 


Ideas 


Experimental 


2.54 


3.38 




Control 


2.68 


2.75 


Organization 


Experimental 


2.60 


3.15 




Control 


2.61 


2.70 


Voice 


Experimental 


2.73 


3.60 




Control 


2.91 


3.12 


Word Choice 


Experimental 


2.73 


3.26 




Control 


2.91 


3.11 


Sent. Fluency 


Experimental 


2.85 


3.12 




Control 


2.89 


2.89 


Conventions 


Experimental 


2.79 


2.98 




Control 


2.99 


2.99 



Table 1 shows that pretest scores for the experimental and control groups are very similar on all 
traits. The treatment group gained the most on those traits receiving the most emphasis in 
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instruction. Performance on other traits gained slightly. Students in the control group gained 
slightly on two traits, and remained essentially the same on the other four. 

Table 2 shows that gains between the experimental and control groups were significantly diffi 
for Ideas, approach significance for the trait of Organization, tended toward significance for tl 
trait of Voice and were nonsignificant for the remaining traits. 





Table 2 
Analysis of Variance Results 








Trait 


Source of Var 


SS 


DF 


MS 


F Significance 


Ideas 


Within+Residual 
Group 


126.44 
3.84 


131 
1 


.97 
3.84 


3.98 


.048 


Org. 


Within +Residual 
Group 


128.23 
3.01 


131 
1 


.98 
3.01 


3.08 


.082 


Voice 


Within+Residual 
Group 


114.94 
1.4 


131 
1 


.88 
1.4 


1.59 


.209 


Words 


Within+Residual 
Group 


60.85 
.02 


131 
1 


.46 
.02 


.04 


.839 


Sentences 


Within+Residual 
Group 


117.29 
.60 


131 
1 


.90 
.60 


.67 


.414 


Conven. 


Within+Residual 
Group 


177.79 
.72 


131 
1 


1.36 
.72 


.53 


.466 



Discussion 

It is interesting to note that student scores improved on traits in proportion to the amount of time 
spent on them and the order in which they were introduced. Thus, students showed the most 
improvement on Ideas, the trait introdu d first; an almost significantly different gain on 
Organization, the trait introduced second; and a slightly lower gain on Voice, the trait introduced 
third There was no significant difference in gains on the three traits which were not directly 
taught to students. This certainly lends credibility to the premise that student writing improves to 
the extent that we address instruction to the features of writing we deem to be the most 
important, and to the extent that we directly teach students what good and poor writing looks like 
on each of these dimensions. 
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One possible objection to the study design might be, "Of course students in the experimental 
group do better than those in the control group; they were directly instructed on the traits used in 
the test. We know that direct instruction on the test will improve scores. The real question is 
whether writing improves." The response to this poses the counter-question, "And how do we 
get this external measure of the extent to which writing improves?" The only possible way is to 
make a direct professional judgment of whether writing has improved. We contend that the six- 
trait model is the best available means of making these judgments. If, in fact the six-trait 
analytical model does define what we mean by good writing, then improvements on the traits is a 
direct measure of improvement in writing. The direct, professional judgment of the quality of 
writing is what the six-trait model is all about. There is no better outside measure. 

Therefore, this study does not involve "teaching to the test," it involves "teaching to the criteria " 
Therein lies the power and, in fact, the whole point of the technique of teaching clearly defined 
performance catena directly to students. If, in fact, the six-trait model defines what we value in 
wntmg, then teaching them the traits teaches them, by definition, what good writing is This 
study tends to support the conclusion that, as the result, student writing improves Therefore we 
can impact student achievement by improving classroom assessment techniques teacher skills' in 
using them, and student self-assessment. 
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Appendix A: Scoring Guide Used By Fifuh Grade Students 



IDEAS 




It all makes sense. 
My reader will learn a lot. 



This is what I 
wanted to say. 

Good, juicy details! 



I need to add some details, 
I'm working on it! 



My reader might have some questions. 
There might be more to telL 




My reader won't understand this yet. 
Vm not sure what to say. 
I don 7 really have a topic yet. 
I need more time to think. 
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/ know where I'm going. 
I love my beginning. 
My ending is cooL 



My paper is PRETTY easy 
for a reader to follow. 





Follow me! 



I see just how all the 
parts fit together. 



Maybe I need to move 
some things around. 



I'm not sure how to end it. 



What should I tell First? 
What comes next? 




My reader would feel LOST! 



Help! 

Which fieces go together? 



f don't know where I'm headed. 
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VOICE 




It's ME!! 
This is what I think. 



It might make you laugh or cry. 



I hearaMttfeofme 
in the writing. 






Hear me ROAR! 
Hove this topic 



I want my reader to 
feel what I feel 



V 

This topic is OK. 



Tm starting to have fun. 



I don *t hear myself in this paper. 



This topic is boring. 



SNORE! 
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WORDS 




My reader will love 
the words I chose. 



My words paini a picture. 
My words make (he message CLEAR. 



I like SOME of these words. 





I think this is the 
BEST way to say it! 



Tliese ore words I hue!! 



There is probably a 
BETTER way to say it. 



My reader 
might wonder 
what I meant 



These words are NOT my favorites. 



Some of my words don't make sense 
to me when I read them over. 



4 



I used the first words I thought of. 
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FLUENCY 




My sentences make sense. 
My paper is EASY to read out loud. 



Some sentences are LONG 
and some are SHORT. 



A lot cfmy sentences 
begin the same way. 

I wish my paper sounded 
a little smoother. 





My sentences begin ui 
several different ways. 



I love the sound of this paper. 



My paper is PRETTY 
easy to read out loud 
if you take your time. 



Some of my sentences do not make sense. 

My paper is HARD to read out loud — 
even for me! 

Sometimes, I can't tell where 
to beqin a new sentence. 



4' 
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f'm not happy with the 
sound of this paper yet. 
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CONVENTIONS 




You have to look hard to find 
mistakes in my paper! 

I used capitals to BEGIN SENTENCES. 



My work has been edited and 
proofread; it is EASY to read. 



I'm not sure I spelled 
all the words right. 



I might have left out a period 
or question mark. 



paper has some mistakes, 
but you can still read it. 

I have more editing to do. 





I used capitals 
on NAMES. 



I checked my spelling. 



fused periods (.) and question 
marks (?) in the right spots. 



9 reader would notice 
lots of mistakes. • 



/ have not edited my work yet. 



I need to correct some things 
before I publish this work. 




Appendix B: Scoring Guide Used By Adult Raters 

IDEAS AND CONTENT 

(Development) 

This paper is clear and focused. It holds the reader's attention. 
Relevant anecdotes and details enrich the central theme or storyline. 
o Ideas are fresh and original. 

o The writer seems to be writing from knowledge or experience and shews 
insight: an understanding of life and a knack for picking out what is 
significant. 

o Relevant, telling, quality details give the reader important information 

that goes beyond the obvious or predictable, 
o The writer develops the topic in an enligh^ning, purposeful way that 

makes a point or tells a story, 
o Every piece adds something to the whole. 



The writer is beginning to define the topic, even though development 
is still basic or general 

o It is pretty easy to see where the writer is headed, though more 

information is needed to "fill in the blanks." 
o The writer seems to be drawing on knowledge or experience, but has 

difficulty going from general observations to specifics, 
o Ideas are reasonably clear, though they may not be detailed, personalized, 

or expanded enough to show indepth understanding or a strong sense of 

purpose. 

o Support is attempted, but doesn't go far enough yet in fleshing out the 

main point or storyline, 
o Details often blend the original with the predictable. 

As yet, the paper has no clear sense of purpose or central theme. To 
extract meaning from the text, the reader must make inferences 
based on sketchy details. The writing reflects more than one of 
these problems: 

o The writer is still in search of a topic, or has not begun to define the topic 

in a meaningful, personal way. 
o Information is very limited or unclear, 
o The text may be repetitious, or may read like a collection of 

disconnected, random thoughts, 
o Everything seems as important as everything else; the reader has a hard 

time sifting out what's critical. 
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ORGANIZATION 
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The organization enhances and showcases the central idea or 
storyline. The order, structure or presentation of information is 
compelling and moves the reader through the text 
o Details seem to fit where they're placed; sequencing is logical and 
effective. 

o An inviting introduction draws the reader in; a satisfying conclusion 

leaves the reader with a sense of resolution, 
o Pacing is well controlled; the writer knows when to slow down and 

elaborate, and when to pick up the pace and move on. 
o Thoughtful transitions clearly show how ideas connect, 
o Organization flows so smoothly the reader hardly thinks about it. 

The organizational structure is strong enough to move the reader 

through the text without undue confusion. 

o The paper has a recognizable introduction and conclusion. The 

introduction may not create a strong sense of anticipation; the conclusion 

may not tie up all loose ends, 
o Sequencing is usually logical, but may sometimes be so predictable that the 

structure takes attention away from the content, 
o Pacing is fairly well controlled, though the writer sometimes spurts ahead 

too quickly or spends too much time on details that do not matter, 
o Transitions often work well; at other times, connections between ideas 

are fuzzy. 

o The organization sometimes supports the main point or storyline; at 

other times, the reader feels an urge to slip in a transition or move things 
around. 

The writing lacks a clea, sense of direction. Ideas, details or events 
seem strung together in a loose or random fashion-or else there is 
no identifiable internal structure. The writing reflects more than 
one of these problems: 
o Sequencing needs work. 

o There is no real lead to set up what follows, no real conclusion to wrap 
things up. 

o Pacing f*els awkward; the writer slows to a crawl when the reader wants 

to get on with it, and vice versa, 
o Connections between ideas are confusing or missing, 
o Problems with organization make it hard for the reader to get a grip on 

the main point or storyline. 
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VOICE 



The writer speaks directly to the reader in a way that is 
individualistic, expressive and engaging. Clearly, the writer is 
involved in the text, is sensitive to the needs of an audience, and is 
writing to be read. 

o The reader feels a strong interaction with the writer, sensing the person 
behind the words* 

o The tone and voice give flavor to the message and seem appropriate for 

the purpose and audience, 
o Narrative writing seems honest, appealing, and written from the heart 
o Expository or persuasive writing reflects a strong commitment to the 

topic, and an effort to bring the topic to life by anticipating the reader's 

questions, and showing why the reader should care or want to know 

more. 

The writer seems sincere, but not fully engaged or involved The 

result is pleasant or even personable, but not compelling. 

o The writing communicates in an earnest, pleasing manner. Moments here 

and there surprise, amuse or move the reader, 
o Voice may emerge strongly on occasion, then retreat behind general, 

dispassionate language* 
o The writing hides as much of the writer as it reveals, 
o The writer seems aware of an audience, but often to weigh words 

carefully or discard personal insights in favor of safe generalities. 

The writer seems indifferent, uninvolved or distanced from the topic 
and/or the audience. As a result, the writing is lifeless or 
mechanical; depending on the topic, it may be overly technical or 
jargonistic. The paper reflects more than one of the following 
problems: 

o It is hard to sense the writer behind the words. The writer does not 

seem to reach out to an audience, or to anticipate their interests and 

questions. , 
o The writer speaks in a kind of monotone that flattens all potential highs or 

lows of the message, 
o The writing may communicate on a functional level, but it does not move 

or involve the reader, 
o The writer does not seem sufficiently at home with the topic to take 

risks, share personal insights or make the topic/story personal and real for 

the reader. 
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WORD CHOICE 



Words convey the intended message in a precise, interesting and 
natural way. 

o Words are specific and accurate; it is easy to understand just what the 
writer means. 

o The language is natural and never overdone; phrasing is highly 
individual. 

o Lively verbs energize the writing Precise nouns and modifiers create 
pictures in the reader's mind. 

o Striking words and phrases often catch the reader's eye--and linger in the 
reader's mind. 

o Cliches and jargon are used sparingly, only for effect. 



The language is functional, even if it lacks punch; it is easy to 
figure out the writer's meaning on a general level 
o Words are almost always correct and adequate; they simply lack flair, 
o Familiar words and phrases communicate, but rarely capture the reader's 

imagination. Still, the paper may have one or two fine moments, 
o Attempts at colorful language come close to the mark, but sometimes 

seem overdone. 

o Energetic verbs or picturesque phrases liven things up now and then; the 
reader longs for more. 



The writer struggles with a limited vocabulary, searching for words 
to convey meaning. The writing reflects more than one of these 
problems: 

o Language is so vague (e.g., // was a fun time, She was neat, It was nice, 
We did lots of stuff) that only the most general message comes through, 
o Persistent redundancy distracts the reader, 
o Jargon or cliches serve as a crutch. 

o Words are used incorrectly, sometimes making the message hard to 
decipher. 

o Problems with language leave the reader wondering what the writer is 
trying to say. 
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SENTENCE FLUENCY 
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The writing has an easy flow and rhythm when read aloud 
Sentences are well built, with strong and varied structure that invites 
expressive oral reading. 

o Sentences are constructed in a way that helps make meaning clear, 
o Purposeful sentence beginnings show how each sentence relates to and builds 
upon the one before it. 

o The writing has cadence, as if the writer has thought about the sound of the words 

as well as the meaning, 
o Sentences vary in length as well as structure, 
o Fragments, if used, add style, 
o Dialogue, if used, sounds natural. 

Vie text hums along with a steady beat, but tends to be more 
pleasant or businesslike than musical, more mechanical than fluid. 

o Sentences may not seem artfully crafted or musical, but they are usually 

grammatical. They hang together. They get the job done, 
o There is at least some variation in sentence length and structure. Sentence 

beginnings are NOT all alike, 
o The reader sometimes has to hunt for clues (e.g., connecting words and phrases 

like however, therefore, naturally, after a while, on the other hand, to be 

specific, for example, next, first of all, later, but as it turned out, although, etc.) 

that show how sentences interrelate, 
o Parts of the text invite expressive oral reading; others may be stiff, awkward, 

choppy or gangly. Overall though, it is pretty easy to read the paper aloud with a 

little practice. 

The reader has to practice quite a bit in order to give 

this paper a fair interpretive reading. The writing reflects more than 

one of the following problems: 

o Sentences are choppy, incomplete, rambling or awkward; they need work, 
o Phrasing does not sound natural, the way someone might speak. The reader 

must sometimes pause or read over to get the meaning, 
o Many sentences begin the same way— and may follow the same patterns (e.g., 

subject-verb-object) in a monotonous pattern, 
o Endless connectives {and, and so, but then, because, and then, etc.) 

create a massive jumble of language in which clear sentence beginnings and 

endings get swallowed up. 
o The text does not invite expressive oral reading. 
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CONVENTIONS 



The writer demonstrates a good grasp of standard writing conventions 
fag * grammar, capitalization, punctuation, usage, spelling, paragraphing) 
and uses conventions effectively to enhance readability. Errors tend to be 
so few and so minor that the reader can easily overlook them unless 
hunting for them specifically. 

o Paragraphing tends to be sound and to reinforce the organizational structure, 
o Grammar and usage are correct and contribute to clarity and style, 
o Punctuation is accurate and guides the reader through the text, 
o Spelling is generally correct, even on more difficult words, 
o The writer may manipulate conventions-especially grammar and spelling-for 
stylistic effect. 

o GRADES 7 AND UP ONLY; The writing is sufficiently long and complex to 

allow the writer to show skill in using a wide range of conventions, 
o Qalyjight editing would be required to polish the text for publication. 

The writer shows reasonable control over a limited range of standard 
writing con ventions. Con ventions are sometimes h an died well an d 
enhance readability; at other times, errors are distracting and impair 
readability. 

o Paragraphing is attempted. Paragraphs sometimes run together or begin in the 
wrong places. 

o Problems with grammar or usa^e are not serious enough to distort meaning, 
o Terminal (end-of sentence) punctuation is usually correct; internal punctuation 

(commas, apostrophes, semicolons, dashes, colons, parentheses) is sometimes 

missing or wrong. 

o Spelling is usually correct or reasonably phonetic on common words, 
o Moderate editing would be required to polish the text for publication. 

Errors in spelling, punctuation, usage and grammar, capitalization, 
and/or paragraphing repeatedly distract the reader and make the text 
difficult to read The writing reflects more than one of these problems: 
o Paragraphing is missing, irregular, or so frequent (e.g., every sentence) that it 

has no relationship to the organizational structure of the text, 
o Errors in grammar or usage are very noticeable, and may affect Extensive 

editing would be required to polish the text for publication.meaning. 
o Punctuation (including terminal punctuation) is often missing or incorrect, 
o Spelling errors are frequent, even on common words, 
o The reader must read once to decode, then again for meaning, 
o Extensive editing would be required to polish the text for publication. 
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